Predictive information and Bayesian surprise in exchangeable random processes
نویسندگان
چکیده
In this report we examine the connection between Itti and Baldi’s ‘Bayesian suprise’ [1] and instantaneous predictive information [2], and show that the two are identical in certain classes of exchangeable random sequences. We also examine how these relationships can be used to compute the instantaneous predictive information in a Markov chain where the transition matrix is unknown. Samples from such a process are not exchangable, but nonetheless useable results can be obtained. Predictive information and Bayesian surprise in exchangeable random processes Samer Abdallah ([email protected]) Mark Plumbley ([email protected]) 31st October, 2010 Abstract In this report we examine the connection between Itti and Baldi’s ‘Bayesian suprise’ [1] and instantaneous predictive information [2], and show that the two are identical in certain classes of exchangeable random sequences. We also examine how these relationships can be used to approximate the instantaneous predictive information in a Markov chain where the transition matrix is unknown even though samples from such a process are not exchangable, showing that the Bayesian surprise remains as one of two components of the instantaneous predictive information.In this report we examine the connection between Itti and Baldi’s ‘Bayesian suprise’ [1] and instantaneous predictive information [2], and show that the two are identical in certain classes of exchangeable random sequences. We also examine how these relationships can be used to approximate the instantaneous predictive information in a Markov chain where the transition matrix is unknown even though samples from such a process are not exchangable, showing that the Bayesian surprise remains as one of two components of the instantaneous predictive information.
منابع مشابه
Predictive Information in Gaussian Processes with Application to Music Analysis
We describe an information-theoretic approach to the analysis of sequential data, which emphasises the predictive aspects of perception, and the dynamic process of forming and modifying expectations about an unfolding stream of data, characterising these using a set of process information measures. After reviewing the theoretical foundations and the definition of the predictive information rate...
متن کاملProvide a Predictive Model to Identify People with Diabetes Using the Decision Tree
Background: Today, in most hospitals in Iran, there is an extensive database of patient characteristics that includes a large amount of information related to medical, family and medical records. Finding a knowledge model of this information can help to predict the performance of the medical system and improve educational processes. Methods: Data mining techniques are analytical tools that are...
متن کاملInformation theory of Exchangeable Estimators
Exchangeable random partition processes are the basis for Bayesian approaches to statistical inference in large alphabet settings. On the other hand, the notion of the pattern of a sequence provides a framework for data compression in large alphabet scenarios. Because data compression and parameter estimation are intimately related, we study the redundancy of Bayes estimators coming from Poisso...
متن کاملExchangeable Random Processes and Data Abstraction
Several Turing-complete probabilistic programming languages such as Church, Venture and Anglican support so-called exchangeable random processes (XRPs) as built-in abstract data types. Instances of these data types denote infinite sequences of random values. Every such object satisfies the so-called exchangeability property, which says that the probability distribution of the object does not ch...
متن کاملBeta-Negative Binomial Process and Exchangeable Random Partitions for Mixed-Membership Modeling
The beta-negative binomial process (BNBP), an integer-valued stochastic process, is employed to partition a count vector into a latent random count matrix. As the marginal probability distribution of the BNBP that governs the exchangeable random partitions of grouped data has not yet been developed, current inference for the BNBP has to truncate the number of atoms of the beta process. This pap...
متن کامل